This invention relates to a method and an apparatus for retrieving moving image, which retrieve and identify fixed moving image from some moving image. And more particularly to, a method and an apparatus for retrieving moving image, in which the information of the unique identification of moving image which depends on the content of the moving image is extracted and registered, the registered moving image is retrieved and identified earlier than video rate from moving image such as moving image archives sent in broadcasting and the like.
It is well known that there are various technologies on the international standard encoding system such as MPEG (Motion Picture Experts Group) as the information compressing technology for storing and transmitting moving image. It becomes possible that moving image is sufficiently handled even in the PC (Personal Computer), if the encoded moving image compressed by this international standard encoding system is used. The technology for monitoring the specific moving image by using the PC, for example monitoring a television commercial (identifying moving image), is ever studied and is developed.
As the conventional method for retrieving moving image, there are several methods for retrieving moving image, for example a method using the keyword retrieval, a method using the visual discrimination by the visual observation of the user, and a method using the number of continuous frames on the shot of moving image disclosed in Japanese Patent Publication No. JP7-46517.
According to the method using the keyword retrieval, moving image is given a classification code, a title, the content of the moving image, various relation information and so on as the information of keyword index, then the moving image is retrieved and identified by retrieving this keyword.
And, according to the method using the visual discrimination by the visual observation of the user, the user memorizes moving image that the user intends to retrieve and to identify then the moving image is retrieved and identified by the user inspecting visually all moving images.
And, according to the apparatus disclosed in Japanese Patent Publication No. JP7-46517, the number of continuous frames on each shot is searched between cut points by the means for detecting cut point, then moving image is retrieved by using the column of the number of continuous frames as index information.
And, in the real time discrimination method of image scene based on the time series coding of cutting (Nagasaka others, April, 1996 Inst. of Electronics, Inf. and Commun. Engineering paper magazine D-II Vol.J79-D-II No.4 pp. 531-537), the frame image of a cut point is extracted, and average color information is calculated by dividing the frame into 2xc3x972 and encoding, and the encoded information is arranged in the time series and applied to the discrimination (retrieval) of image scene.
And, there is an apparatus for retrieving the cut point of moving image disclosed in Japanese Patent Publication No. JP7-236153 as an apparatus for retrieving moving image by detecting the cut point of the moving image. In this apparatus for retrieving the cut point of moving image disclosed in Japanese Patent Publication No. JP7-236153, the time variation of luminance difference between frames of the reduction image which is made from prepared reduction image and the mean value component data of the information of compressed and encoded moving image and the time variation of color difference histogram correlation value are used in order to obtain a cut point at high speed from moving image.
And, there is an apparatus for processing moving image disclosed in Japanese Patent Publication No. JP7-152779 as an apparatus for detecting the characteristic frame of moving image. In this apparatus for processing moving image disclosed in Japanese Patent Publication No. JP7-152779, the low frequency part of DCT (Discrete Cosine Transform) component is used in order to detect characteristic frame that is used for as index information (retrieving information), for example a cut point frame, a motionless frame, a character frame, a specific color frame, etc., from compressed moving image data including the DCT operation.
In the conventional methods and apparatuses for retrieving moving image, however, there are disadvantages as follows.
That is to say, in the conventional method for retrieving moving image by the keyword retrieval, there is a disadvantage that the considerable cost for registering keyword index information is taken, because to beforehand register the keyword index information is necessary. And more, there is a disadvantage with the case in which keyword index information differs even in the case of retrieving of identical moving image by the user, because the feeling of moving image of each user who retrieves moving image and the way of understanding are different. Moreover, there is a disadvantage that the time for choosing the keyword index of the desire is taken, even the system for inputting keyword index in which the user who retrieves moving image from keyword index information presented by a list type is made to choose the keyword index of the desire.
And, in the conventional method for retrieving moving image using the visual discrimination by the visual observation of the user, there is a disadvantage that hands are taken and the cost is high, because it is the method in which the user who retrieves moving image memorizes moving image that the user intends to retrieve and identify, and the user retrieves and identifies the moving image of the desire by observing all moving images.
And, in the method for retrieving moving image using the number of continuous frames on the shot which constitutes moving image disclosed in Japanese Patent Publication JP7-46517, there is a disadvantage that it might be misunderstood that moving image is identical with moving image for retrieving, if the number of continuous frames of the moving image agrees even if the content of the moving image differs from the moving image for retrieving, because the number of continuous frames is not index information reflecting the content of the meaning of moving image.
And, in the conventional real time discrimination method of image scene based on the time series coding of cutting, there is a disadvantage that the time for the extracting process of a cut point frame is taken and the cost also rises, because it is necessary to accurately extract a cut point frame in order to obtain a right collation result.
And, in the apparatus for retrieving the cut point of moving image disclosed in Japanese Patent Publication No. JP7-236153, there is a disadvantage that it is impossible to retrieve and identify moving image of the desire automatically, because the detection of a cut point in moving image is a purpose, and it is only to detect a cut point by regarding the time variation of luminance difference and the time variation of color difference histogram correlation value.
In the apparatus for processing moving image disclosed in Japanese Patent Publication No. JP7-152779, there is a disadvantage that time is taken due to the complicated process and the cost is high. Because the interim result which corresponds to the result of DCT operation by the process for reconstructing image data from compressed image data is made to be signals of the information of the feature quantity of moving image index, at least a kind of moving image index is detected from this signals of the information of the feature quantity of moving image index, and the sum of difference value with the moving image index of examined image is processed at the threshold, and then moving image is detected as index image by the result of this threshold processing.
Accordingly, it is an object of the invention to provide a method and an apparatus for retrieving moving image, in which the information of the unique identification of moving image used for retrieving in accordance with the content of the moving image can be automatically extracted even in the computer with small resources such as PC, and in which moving image can be retrieved and identified automatically by using the information of the unique identification of moving image.
According to the first feature of the invention, an apparatus for retrieving moving image to retrieve pre-determined moving image from object moving image, comprises first extracting means for extracting the information of the feature quantity of each frame from the pre-determined moving image, second extracting means for extracting the information of the unique identification of moving image to identify the moving image in accordance with the information of the feature quantity, extracted by the first extracting means, and retrieving means for retrieving the pre-determined moving image from the object moving image by comparing the information of the unique identification, extracted by the second extracting means, with the information of the unique identification prepared in accordance with the object moving image.
According to the second feature of the invention, a method for retrieving moving image to retrieve pre-determined moving image from object moving image, comprises the steps of (A) extracting the information of the feature quantity of each frame from the pre-determined moving image, (B) extracting the information of the unique identification of moving image to identify the moving image in accordance with the information of the feature quantity, extracted by said step (A), and (C) retrieving the pre-determined moving image from the object moving image by comparing the information of the unique identification, extracted by the step (B), with the information of the unique identification prepared in accordance with the object moving image.
According to the third feature of the invention, a program for retrieving moving image to retrieve pre-determined moving image from object moving image, which is executed by computer system, comprises the steps of (A) extracting the information of the feature quantity of each frame from the pre-determined moving image, (B) extracting the information of the unique identification of moving image to identify the moving image in accordance with the information of the feature quantity, extracted by the step (A), and (C) retrieving the pre-determined moving image from the object moving image by comparing the information of the unique identification, extracted by the step (B), with the information of the unique identification prepared in accordance with the object moving image.